Includes infromation on each court case, such as:
ds.cases_stats
| counts | percentages | ||
|---|---|---|---|
| win side | for petitioner | 284.0 | 66.510539 |
| for respondent | 143.0 | 33.489461 | |
| cases | 427.0 | NaN | |
| courts | 1.0 | NaN | |
| years (2014 to 2019) | 6.0 | NaN | |
| petitioners | 413.0 | NaN | |
| respondents | 356.0 | NaN |
Includes infromation on each speaker, such as:
ds.speakers_stats
| counts | percentages | ||
|---|---|---|---|
| speaker type | advocate (A) | 8942.0 | 99.610115 |
| justice (J) | 35.0 | 0.389885 | |
| speaker names | 8928.0 | NaN | |
| speaker keys | 8977.0 | NaN |
Includes infromation on each vote and voter, such as:
ds.voters_stats.head(7)
| counts | percentages | ||
|---|---|---|---|
| votes | for petitioner | 1912.0 | 60.659898 |
| for respondent | 1240.0 | 39.340102 | |
| justices | 11.0 | NaN | |
| justice | j__john_g_roberts_jr | 363 | 0.661157 |
| j__antonin_scalia | 66 | 0.651515 | |
| j__anthony_m_kennedy | 240 | 0.658333 | |
| j__clarence_thomas | 364 | 0.532967 |
Includes infromation on each advocate (non-judges), such as:
ds.advocates_stats
| counts | percentages | ||
|---|---|---|---|
| side | for petitioner | 403.0 | 50.124378 |
| for respondent | 401.0 | 49.875622 | |
| total advocates | 391.0 | NaN | |
| total roles | 154.0 | NaN | |
| aggregate roles | inferred | 10.0 | 1.243781 |
| for respondent | 404.0 | 50.248756 | |
| for petitioner | 390.0 | 48.507463 |
Includes infromation on each conversation. There is one conversation per case,
and conversations are made up of individual utterances. Conversation information includes:
conversations.head(2)
| id | case_id | winning_side | |
|---|---|---|---|
| 0 | 23291 | 2014_13-553 | 1 |
| 1 | 23252 | 2014_13-895 | 1 |
Includes infromation on each utterance, such as:
cols = ["case_id", "speaker", "speaker_type", "conversation_id", "text"]
utterances.head(2).loc[:, cols]
| case_id | speaker | speaker_type | conversation_id | text | |
|---|---|---|---|---|---|
| 0 | 2014_13-553 | j__john_g_roberts_jr | J | 23291 | we'll hear argument next in case no. 13-553, t... |
| 1 | 2014_13-553 | andrew_l_brasher | A | 23291 | thank you, mr. chief justice, and may it pleas... |
spaCy)['a', 'the', 'by'])cases_proc.head(2)
| case_id | tokens | avg_num_sentences | avg_num_words | year | win_side | |
|---|---|---|---|---|---|---|
| 0 | 2014_13-553 | [hear, argument, case, alabama, department, re... | 2.447368 | 178.478947 | 2014 | 1 |
| 1 | 2014_13-895 | [hear, argument, case, number, alabama, legisl... | 2.432203 | 184.368644 | 2014 | 1 |
In addition to standard columns, includes columns on count of advocates for petitioner or respondent.
judges_proc.head(2)
| case_id | tokens | avg_num_sentences | avg_num_words | year | win_side | |
|---|---|---|---|---|---|---|
| 0 | 2014_13-553 | [hear, argument, case, alabama, department, re... | 1.682692 | 94.5000 | 2014 | 1 |
| 1 | 2014_13-895 | [hear, argument, case, number, alabama, legisl... | 2.039062 | 132.9375 | 2014 | 1 |
advocate_proc.head(2)
| case_id | tokens | avg_num_sentences | avg_num_words | year | win_side | |
|---|---|---|---|---|---|---|
| 0 | 2014_13-553 | [hear, argument, case, alabama, department, re... | 2.212329 | 163.547945 | 2014 | 1 |
| 1 | 2014_13-895 | [mr, chief, justice, court, alabama, employ, r... | 2.326241 | 164.439716 | 2014 | 1 |
adversary_proc.head(2)
| case_id | tokens | avg_num_sentences | avg_num_words | year | win_side | |
|---|---|---|---|---|---|---|
| 0 | 2014_13-553 | [handpicked, business, transport, good, motor,... | 3.227273 | 228.022727 | 2014 | 1 |
| 1 | 2014_13-895 | [hear, argument, case, number, alabama, legisl... | 2.589474 | 213.947368 | 2014 | 1 |
Logistic regression is a binary classification model that predicts court case outcomes based on a 'bag of words'.
It assumes a linear relationship between variables and doesn't consider precedents or social trends.
The current model did not use regularization, which is used to help prevent overfitting.
Four datasets were used, and models based on advocate and adversary utterances achieved higher accuracies than those using judge utterances or a combination of all utterances.
The logistic regression model provided insights into the predictive power of different utterances, indicating that advocate and adversary statements are more predictive than the more than judge utterances, and all utterances aggregated together for case outcomes.
XGBoost, short for eXtreme Gradient Boosting, is an ensemble model that uses gradient boosting with decision trees to minimize the loss function.
It sequentially grows trees, considering the residuals of the previous tree and reweighting the observations.
Unlike Random Forest, XGBoost adjusts the model on every iteration using the previous residuals as the new target variable, allowing it to learn from mistakes and improve.
Limitations of XGBoost include difficulty in interpretation due to its use of multiple trees and its predisposition to overfitting if parameters are not tuned properly.
XGBoost performs better than single models but requires a finetuning process to determine the best hyperparameters for a specific context.
Random Forest is an ensemble model with multiple decision trees that combines the bagging and random feature selection methods.
Limitations of the Random Forest model include reduced interpretability compared to decision trees and the need for more time and resources for training due to bagging and random feature subsets.
The predictions based on the bag of words CountVectorizer solely consider word frequency and may not capture complex linguistic relationships.
The Random Forest model was chosen to capture complex interactions in unstructured data, avoid overfitting, and rank word importance.
You can assess the model's accuracy by predicting case outcomes and examining its word importance metrics.
Cross-validation and testing on out-of-sample data help to reveal how well the model generalizes and avoids overfitting.
simple_bar_plot(xlabel, ylabel, labs, y)
disp_accuracy(lr_acc, labs=list(lr_acc["dataset"]))
disp_conf_matrix(lr.confusion_matrix["case_aggregations"])
disp_conf_matrix(lr.confusion_matrix["judge_aggregations"])
disp_conf_matrix(lr.confusion_matrix["advocate_aggregations"])
disp_conf_matrix(lr.confusion_matrix["adversary_aggregations"])
plot_accuracy_scores(
max_iter_f1_melted, maxiter, lr_label, accuracy_metric="F1"
)
plot_accuracy_scores(max_iter_melted, maxiter, lr_label)
plot_accuracy_scores(et_df, maxiter, lr_label, accuracy_metric="Execution Time")
plot_accuracy_scores(max_feature_lr_melted, maxfeat, lr_label)
plot_accuracy_scores(
max_feature_f1_lr_melted, maxfeat, lr_label, accuracy_metric="F1"
)
plot_accuracy_scores(et_df, maxfeat, lr_label, accuracy_metric="Execution Time")
disp_accuracy(rf_acc, labs=list(rf_acc["dataset"]))
disp_conf_matrix(rf.confusion_matrix["case_aggregations"])
disp_conf_matrix(rf.confusion_matrix["judge_aggregations"])
disp_conf_matrix(rf.confusion_matrix["advocate_aggregations"])
disp_conf_matrix(rf.confusion_matrix["adversary_aggregations"])
plot_accuracy_scores(max_depth_rf_melted, maxdepth, rf_label)
plot_accuracy_scores(
max_depth_f1_rf_melted,
maxdepth,
rf_label,
accuracy_metric="F1",
)
plot_accuracy_scores(
et_df,
maxdepth,
rf_label,
accuracy_metric="Execution Time",
)
plot_accuracy_scores(max_feature_melted, maxfeat, rf_label)
plot_accuracy_scores(
max_feature_f1_melted, maxfeat, rf_label, accuracy_metric="F1"
)
plot_accuracy_scores(et_df, maxfeat, rf_label, accuracy_metric="Execution Time")
plot_accuracy_scores(num_trees_melted, ntree_lab, rf_label)
plot_accuracy_scores(
num_trees_f1_melted,
ntree_lab,
rf_label,
accuracy_metric="F1",
)
plot_accuracy_scores(
et_df,
ntree_lab,
rf_label,
accuracy_metric="Execution Time",
)
disp_accuracy(xg_acc, labs=xg_acc["dataset"])
disp_conf_matrix(xg.confusion_matrix["case_aggregations"])
disp_conf_matrix(xg.confusion_matrix["judge_aggregations"])
disp_conf_matrix(xg.confusion_matrix["advocate_aggregations"])
disp_conf_matrix(xg.confusion_matrix["adversary_aggregations"])
plot_accuracy_scores(et_df, eta, xg_label, accuracy_metric="Execution Time")
plot_accuracy_scores(eta_melted, eta, xg_label)
plot_accuracy_scores(eta_f1_melted, eta, xg_label, accuracy_metric="F1")
plot_accuracy_scores(max_depth_melted, maxdepth, xg_label)
plot_accuracy_scores(
max_depth_f1_melted, maxdepth, xg_label, accuracy_metric="F1"
)
plot_accuracy_scores(
et_df, maxdepth, xg_label, accuracy_metric="Execution Time"
)
# Plot accuracy scores
plot_accuracy_scores(subsample_melted, subsamp, xg_label)
plot_accuracy_scores(
subsample_f1_melted, subsamp, xg_label, accuracy_metric="F1"
)
plot_accuracy_scores(et_df, subsamp, xg_label, accuracy_metric="Execution Time")
disp_accuracy(model_comp, labs=model_comp["model"])